
Computer Graphics, Volume 22, Number 4, August 1988 



Reconstruction Filters in Computer Graphics 

Don P. Mitchell 

Aran N. Netravali 

AT&T Bell Laboratories 
Murray ffiU, New Jersey 07974 



ABSTRACT 



Problems of signal processing arise in image 
synthesis because of transformations between 
continuous and discrete representations of 2D 
images. Aliasing introduced by sampling has 
received much attention in graphics, but recon- 
struction of samples into a continuous 
representation can also cause aliasing as well 
as other defects in image quality. The prob- 
lem of designing a filter for use on images is 
discussed, and a new family of piecewise 
cubic filters are investigated as a practical 
demonstration. Two interesting cubic filters 
are found, one having good antialiasing pro- 
perties and the other having good image- 
quality properties. It is also shown that recon- 
struction using derivative as well as amplitude 
values can greatly reduce aliasing. 
CR Categories and Subject Descriptions: 1.3.3 [ Computer Graphics ]: 
Picture/Image Generation; 1.4.1 [ Image Processing ]; Digitization 

General Terms: Algorithms 

Additional Keywords and Phrases: Antialiasing, Cubic Filters, Filters, 
Derivative Reconstruction, Reconstruction, Samplmg 

1. Introduction 

The issues of signal processing arise in image synthesis because of 
transformation between continuous and discrete representations of 
images. A continuous signal is converted to a discrete one by sampling, 
and according to the sampling theorem [SHA49], all the information in 
the continuous signal is preserved in the samples if they are evenly 
spaced and the frequency of sampling is twice that of the highest fre- 
quency contamed in the signal. A discrete signal can be converted to a 
continuous one by interpolating between samples, a process referred to in 
the signal-processing Mterature as reconstruction. 



Many conversions between continuous and discrete representations may 
occur in the course of generating an image. For example when ray trac- 
ing a texture-mapped surface, a photograph may be sampled by a digi- 
tizer to define the texture, then the texture samples are interpolated and 
resampled when a ray strikes the textured surface, the ray samples are 
interpolated and resampled to generate pixel values, and the pixels are 
interpolated by a display and finally resampled by retinal cells when the 
image is viewed. Resampling may be more explicit, as in enlarging or 
reducing a digital image or warping an image (e.g., with Catmull and 
Smith's algorithm [CAT80]). Each of these conversions can introduce 
conspicuous errors mto an image. 

Errors introduced by samphng (e.g., aliasing) have received considerable 
attention in the graphics community since Crow identified this as the 
cause of certain unwanted artifacts in synthetic images [CR077]. Alias- 
ing in images was discussed in the classic 1934 paper by Mertz and Gray 
[MER34]. Their discussion contams a description of artifacts well- 
known to graphics researchers today and shows that the condition for 
preventing aliasing was known, as a rule of thumb, long before 
Shannon's proof of the samphng theorem: 

The interference usually manifests itself in the form of serra- 
tions on diagonal lines and occasional moire effects in the 
received picture. Confusion in the signal may be practically 
eliminated by using an aperture of such a nature that it cuts 
off all [Fourier] components with n numbers greater than 
N/2 [half the scanning rate] .... 

By comparison, the problems introduced by reconstruction have been 
somewhat neglected in the graphics literature. Reconstruction can be 
responsible for aliasing and other types of distortion that mar the subjec- 
tive quality of an image. This paper will focus on the effects of recon- 
stiTiction and how to design filters for graphics appUcations. 

2. Aliasing Caused by Reconstruction 

Aliasing in synthetic images is a serious problem and still not completely 
solved. In otiier digital-signal-processing applications, aliasing is eMm- 
inated by prefiltering signals before sampUng, as iUusttated m Figure 1. 
Note tiiat it is tiie prefUtered signal that is reconstiiicted in tiiis case. 
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Figure 1. Sampling and Reconstniction 



While prefiltering is the classic solution to abasing problems, there is a 
special problem encountered in computer graphics. Many synthetic 
©1988 ACM-0-89791-275-6/88/008/0221 $00.75 images originate from what we will call proceduralsignals, in which tiie 
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signal is only implicitly defined by an algorithm for computing point 
samples. Operations that require an explicit representation of the signal 
cannot be performed, and in particular, prefiltering is impractical. This 
drSiculty is unique to computer graphics, and ray tracing is the clearest 
example of it [WfflSO]. 

To explain the role that reconstruction plays m aliasing, it will be helpful 
to review briefly the theory of sampling and define the operations of 
sampling and reconstruction more precisely. In one dimension, a signal 
can be represented by a continuous function f(x). Producing a discrete 
signal by sampling is equivalent to multiplying by an infmite train of 
impulses known as a comb function: 

fAx)=f(x)comb(x) (I) 

where 

comb(x)= 2 SOc-n) (lb) 



centered at the location of an impulse in the comb. Equation (4) states 
that, in the frequency domain, reconstruction can be interpreted as the 
multiplication by K(v) which is intended to eliminate all the extraneous 
replicas of the signal's spectrum and keep the original base-band cen- 
tered at the origm. K(v) is indicated by the dashed curve in Figure 2. 

However, Figure 2 also demonstrates a problem. The replicas of the sig- 
nal spectrum overlap, and the reconstruction filter can not isolate a pure 
version of the base-band signal. When part of the energy in a replica of 
the spectrum leaks into the reconstructed signal, aliasing results. If the 
bandwidth of the signal were narrower or the samphng rate higher, the 
copies would not overlap, and exact reconstruction would be possible. 

Even if the replicated spectra do not overlap, aliasing can result from 
poor reconstruction, as illustrated in Figure 3. When aliasing is a conse- 
quence of undersampling (or lack of prefiltering), it is referred to as 
prealiasing, and when it results from poor reconsttuction, it is called 
postaliasing. 



Unit spacing between samples is assumed in equation (lb), and 5(j:) is 
the Dirac delta function. In this case, the sampling theorem states that 
f(x) can be reconstructed exactly from its samples if it contains no fre- 
quencies greater than 0.5 cycles per sample. This critical frequency is 
called the Nyquist frequency . 

Reconstruction is accomphshed by convolving (indicated by *) the 
discrete signal with a reconstruction filter kernel, k(x): 

Mx) = f,{x)* kix) (2) 
= ]f,(uyk(,x-u)du (2b) 

= i/(n)t(jr-n) (2c) 



Except in the mathematically ideal case, some error is introduced in the 
process of sampling and reconstruction, and/fxj will be somewhat dif- 
ferent from f,(x). To analyze this error, it is useful to view the problem 
in the frequency domain. The Fourier transform of the signal is its spec- 
trum F(v), and the Fourier transform of the filter is its frequency 
response K(v). Since multipMcation in the spatial domain is equivalent 
to convolution in the frequency domain (and vice versa), sampling can 
be described by: 

^'.(v) = f (V) * Comb(y) (3) 
and leconstiuction by: 

/■,(v) = f,(v)-ie(v) (4) 
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Figure 3. Postaliasing Resulting from Poor Reconstruction Filter 

Figure 4 shows an extreme example of aliasing in an image. In this fig- 
ure, the two-dimensional signal, f(x,y) = sin(x^ -Hy ), was sampled on a 
128 X 128 pixel grid. Then, these samples were reconstructed with a 
cubic filter (to be described later in the paper) and resampled to 512 x 
512 pixels. 

The rings on the left side of the image are part of the actual signal, but 
the rings on the right side are Moire' patterns due to prealiasing. In the 
center of the image is a fainter set of concentric rings resulting from pos- 
taliasing. PostaUasing occurred when the discrete image of 128 x 128 
pixels was enlarged to 512 x 512 pixels by resampling. Note that this 
conspicuous postaliasing pattern results from "beating" between the sig- 
nal and its alias. This can also be understood from Figure 3, where it 
can be seen that at the Nyquist frequency (indicated by v^r) the signal's 
spectrum and its nearest replica come close together. Power in the spec- 
trum very near the Nyquist frequency is thus the cause of the most diffi- 
cult type of aMasing to remove from an image. This problem has been 
noted by other graphics researchers [C0087] and by Mertz et al. 
[MER34]. 



The Fourier transform of a comb function is also a comb function (with 
reciprocal spacing between unpulses). 



*:(v) 
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Figure 2. Sampling and Reconstniction in Fiequency Domain 

Figure 2 illustrates the consequences of the convolution in equation (3). 
The spectrum of a sampled signal F,(v) is the sum of an infinite 
sequence of shifted replicas of the original signal's spectrum, each 



Using the same set of samples as m Figure 4, a much better reconstruc- 
tion filter can be applied (a 30-unit-wide windowed sine filter). Figure 5 
demonstrates a dramatic reduction of the postaliasing pattern, but the 
preaMasing is unaffected. The spectrum of this reconstruction filter is 
very close to the ideal step shape shown in Figures 2 and 3. 

3. Other Image Defects Caused by Reconstruction 

Notice in Figure 2, that a reconstruction filter K(v) has two tasks. First 
it must remove the extraneous replicas of the signal spectrum (to prevent 
aliasing). Second, it should pass the original signal base band, but the 
signal can be distorted if this is not done perfectly. This second type of 
reconstruction error will be referred to as base-band attenuation. 

From the previous section, one might assume that the literature of signal 
processing provides a complete solution to the reconstruction problem in 
graphics; however, there is a serious difficulty with the ideal sine filter 
that is not obvious from studying its frequency response. Figure 6 
shows a simple figure reconstructed with the same filter used in Figure 
5. The rippling pattern radiating from the edges is called ringing. Ring- 
ing is strongly suggested by the form of the impulse response of the sine 



222 




Computer Graphics, Volume 22, Number 4, August 1988 



filter, as shown in Figure 7: 



1 - 
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Hguie 7. Impulse Response of Ideal Sine Filter 

Classical digital filter design places a heavy emphasis on the frequency 
response of a filter. That works well in the audio domain, but when con- 
sidering the appearance of images, it is important to also pay attention to 
the shape of the impulse response. 

The response of human viewers to various spatial effects of filters is not 
yet a well-understood science and is largely subjective in nature. Filters 
that have some aliasing problems or certain types of base-band attenua- 
tion may turn out to give visually-pleasing results. Schreiber and Troxel 
have discussed the spatial effects of reconstruction filters [SCH85], and 
they mention some of the important defects that can occur when judging 
the quality of an image subjectively: sample-frequency ripple, anisotro- 
piceffects, ringing, blurring, and aliasing. Each of these effects will be 
considered in detail in the following section. 

Unfortunately, it is often necessary to trade off one type of distortion for 
another, and the design of a smgle filter perfect for aU applications is 
almost certainly impossible. As Figure 6 illustrated, perfect antialiasing 
resulted m the serious defect of ringing. However, Brown realized that a 
moderate amount of ringing can improve the subjective quality of an 
image by enhancing the appearance of sharpness [BR069]. He found 
that a single transient lobe of ringing was effective at sharpening, but 
multiple transients (as in Figure 6) always degrade image quality. 

Many of the concepts presented so far have been illustrated m one 
dimension for simplicity. However, unage reconstruction takes place in 
two dimensions and involves the convolution of a 2D lattice of samples 
with a filter k(x,y). In this paper, we will consider only separable 
filters, where the samples are convolved with the product k(x)k(y). 
Separable filters are computationally more efficient than nonseparable 
because the filtering operation can be performed in separate passes verti- 
cally and horizontally. If the filter kernel is N samples wide, the recon- 
struction can be performed with 0(N^j multiplications for the general 
filter k(x,y) but with 0(N) if the filter is separable. 

4. Piecewise Cubic Reconstruction Filters 

Rather than discuss the issues of filter design abstractly, this paper will 
apply them to the study of a family of filters defined by piecewise cubic 
polynomials. Cubic filters are sufficiently complex to have a broad 
range of behaviors, but they are simple enough to be computationally 
attractive. Hou and Andrews have studied the filtering properties of the 
cubic B-spline [HOU78], and two studies have been made of the one- 
parameter family of cardinal cubic splmes [KEY81,PAR83]. 

The general form for a symmetric cubic filter is: 



*(:t) = 



T\x\' + U\x\^ + V\x\+W iflSI;cl<2 
0 otherwise 



(6) 



number of free parameters. First, the filter should be smooth in the 
sense that its value and first derivative are continuous everywhere. 
Discontinuities in k(x) will lead to high-frequency leakage in the fre- 
quency response of the filter which can allow aliasing. In addition, the 
problem of sample-frequerwy ripple can be designed out of the filter by 
requiring (for all x): 



£ t(jr-ii)= 1 



(7) 



This means that if all the samples are a constant value, the reconstruction 
will be a flat constant signal. Figure 8 demonstrates this defect by using 
an unnormalized Gaussian filter to reconstruct a 512 x 512 image from 
64 X 64 samples. In the frequency domain, sample ripple can be viewed 
as an alias of the image's DC component. It can be shown that the con- 
dition given by equation (7) means that the frequency response of these 
cubic filters will be zero at all integer multiples of the sampling fre- 
quency except zero, eliminating aU extraneous replicas of the DC com- 
ponent. 

With these constramts, the number of free parameters are reduced from 
eight to two, resulting in the following family of cubic filters: 



(l2-9B-6C)\x\'^ it\x\<l 

(-18 + 12B+6C)UI^+(6-2B) 

(-B-6C)lxl'+(6B+30C)lxl^+ iflSI;tl<2 

(-12B-48C)l:tl +(8B+24C) 

0 otherwise 



(8) 



Several obvious constraints can be placed on this function to reduce the 



Some values of (B, C) correspond to well-known cubic spUnes. (1,0) is 
the cubic B-spline, (0, C) is the one-parameter family of cardinal cubics 
with (0, 0.5) being the CatmuU-Rom spline, and (B, 0) are Duffs ten- 
sioned B-spUnes PUF86]. 

In two or more dimensions, visible artifacts can be caused by angle- 
dependent behavior or anisotropic effects. Figure 9 illustrates this prob- 
lem by reconstructing with the separable filter k(x)k(y) using parameter 
values of (0,0). Even though sample-frequency ripple has been designed 
out of k(x), in two dimensions the pixel structure is highly conspicuous 
because the impulse response and the sampling lattice are not radially 
symmetric. 

The phenomenon of ringing has already been seen in Figure 6. Filters 
in the cubic filter family can also exhibit this problem as seen in Figure 
10, where parameter values of (0,1) were used. Ringing results when 
k(x} has negative side lobes, and although some ringing can enhance 
sharpness, a filter that becomes negative is problematic. In Figure 10, a 
typical problem is seen where portions of the image near an edge have 
become negative and have been clamped to zero. This results in pro- 
nounced black spots (e.g., at the top of the statue's head). Similar 
clamping occurs to white, but is less noticeable because of the eye's non- 
linear response to contrast. Schreiber and Troxel have suggested that 
subjectively even sharpening can only be produced by introducing ring- 
ing transients in a suitably nonlinear fashion [SCH85]. These conspicu- 
ous clamping effects could also be eliminated by reducing the dynamic 
range of the image or raising the DC level of the image. 

Parameter values of (3/2, -1/4) result in an image that is unnecessarily 
blurry, as seen in Figure 11. The cubic B-splme also suffers from this 
problem. In viewing many reconsfructions with filters in this family, 
ringing, anisotropy, and blurring are the dominant behaviors, and in a 
small region of the parameter space, a satisfactory compromise seems to 
exist which is seen in Figure 12, using parameter values of (1/3, 1/3). 
This is quite good, considering that the image is being magnified from 
64 x 64 pixels. There is some degree of sharpening, and ahnost no visi- 
ble evidence of the sampling lattice. 

To get a better idea of which regions of the parameter space yield which 
type of behavior, a simple subjective test was designed. On a neutral 
background, four images were displayed typifying the effects of ringing. 
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blurring, anisotropy, and an example of the most satisfactory behavior. 
In the center of the display, images reconstructed from filters with ran- 
dom values of (B, C) were displayed, and the test subject was asked to 
choose which of the four behaviors it exemplified. Nine expert 
observers (researchers working in graphics or image processing) took 
part and over 500 samples were taken. It would not he credible to sug- 
gest that a single ideal parameter pair can be deduced from subjective 
testing. The motivation for this experiment was simply to draw approxi- 
mate boundaries between regions of differing behavior as shown in Fig- 
ure 13. The test subjects were quite consistent with one another in their 
judgements. 
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in Figure 13), quadratic convergence of fit is achieved. This line con- 
tains the cubic B-spline and the CatmuU-Rom spline (which actually has 
cubic convergence). Within the interval of B = 5/3 to B = 0, good sub- 
jective behavior is found with a simple trade-off between blurring and 
ringing. Outside this interval, k{x) becomes bimodal or exhibits extreme 
ringing. The filter (1/3, 1/3) used to generate Figure 12 is recommended 
by the authors, but other observers may prefer more or less ringing. 

5. Postaliasing Revisited 

A systematic consideration of subjective appearance along with quantita- 
tive analysis has yielded an excellent piecewise cubic filter. However, 
the issue of postaUasing, defined in section 2, has been ignored. In fact 
the (1/3, 1/3) filter has only fair antialiasing properties and was used to 
generate Figiire 4. Postaliasing is usually not strong enough to cause 
visible "jaggies" on edges unless a very poor filter is used (e.g., a box 
filter); however, an image with periodic patterns can have conspicuous 
postalias Moire' effects unless careful precautions are taken. Synthetic 
images that contain brick walls, ocean waves, or the ubiquitous checker- 
board pattern are examples of images that might have this difficulty. 
There are several approaches to fixing this problem. 

If the signal is bandlimited and samples carry information about the 
derivative as well as about signal amplitude, a better job of reconstruc- 
tion can be done [PET64]. Given samples (at unit spacing) of a signal 
and of its derivative, a reconstruction can be done in the following form: 



/W= i [/,«(Jt-n)+/./>(Jt-n)] 



(10) 



In an extension of the sampling theorem, if the signal contains no energy 
above the sampling frequency (twice the allowed bandwidth ofsamplmg 
without derivatives), then it can be perfectly reconstructed by the filter 
kernels: 



Hgine 13. Regions of Dominant Subjective Behavior 

To help choose a good filter from the two-parameter space, some quanti- 
tative analysis can be done to remove one more degree of freedom. 
Keys and Park et al. studied the cardinal cubic spUnes because these 
cubics exactly interpolate at the sample positions [KEY81,PAR83]. 
Using standard numerical analysis, Keys concluded that the CatmuU- 
Rom spline was best. Park et al. reached the same conclusion using an 
equivalent analysis in the fi-equency domain. Figure 14 illustrates this 
technique: 



Figure 14. /(jr) and /.(jr) 

As the sample spacing h diminishes, the function and its reconstruction 
become closer. The difference f(x)-f,(x) can be expanded into apower 
series in h to study how parameters affect various orders of behavior. 
Details of this type of analysis can be found in Keys' paper, and when 
applied to the two-parameter family, the following is obtained: 

/C»r)-/,« = (2C+«-l)*/r(*) + 0(/i*) (9) 

r(x) is a polynomial factor. When 2C+B = I (indicated by dotted line 



h(x): 



. 2 



(11) 

(lib) 



This is analogous to the ideal sine reconstruction formula in the standard 
case where no derivative information is present. A common approxima- 
tion to these ideal reconstiuction formulae is Hermite cubic interpolation: 



2Ul'-3UI^-l-i if UlSl 
0 otherwise 



-2irljcl +jr 



if UI<1 
otherwise 



(12) 
(12b) 



Figure 15 shows the aliasing test pattern (still starting with 128 x 128 
samples) reconstructed with the Hermite cubic postfilter. The effect is 
dramatic when compared to Figure 4. The postaliasing artifact in the 
middle of the image is nearly gone, and the preaUasing pattern on the 
right is less mtense. 

The theory of derivative reconstructunmay have some practical value in 
computer graphics. For example, it may be possible to extend Whitted's 
ray-tracing shading model [WH80] to generate derivatives with respect 
to the screen coordinates. This is not an easy problem, but we have 
demonstrated the feasibility of this extension by deriving the formulae 
for Lambert and Phong shading on quadric surfaces. It is possible that 
the density of rays used to reconstruct an image could be reduced in this 
manner by gathering more useful information from each visible surface 
calculation. 

A second approach to improving postaliasing properties is suggested by 
the success of stochastic sampUng on the prealiasing problem 
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[C0086,DIP85,MIT87]. However, preliminary experiments conducted 
by the authors with stochastic-phase reconstruction have yielded very 
poor results. The amount of noise needed to obscure postaliasing seri- 
ously degraded image quality. 

Finally, it was observed in section 2 that signal energy very near the 
Nyquist frequency is most responsible for conspicuous Moire patterns. It 
is possible to cut out this component by notch-filter reconstruction. The 
frequency response of the two-parameter cubic filter in equation (8) is: 

K{v) = |«nc^(v)-«/ic(2v)| (13) 

+ r-3jiiic^(2v)-h2iiiic(2v)+«Jic(4v) 
(jtvr 

+ B sinc*(v) 



This function goes to zero at v = 1/2 when B = 32. In fact, the fre- 
quency response is zero at all integer and half-integer multiples of the 
samphng rate except zero. The filter (3/2, -1/4) is quadratically conver- 
gent, and the result of reconstruction with it can be seen in Figure 16, in 
which the postahasing artifact is almost completely eliminated. Unfor- 
tunately, this filter is quite blurry as was seen in Figure 11. The 
behavior of this notch filter can be seen in its frequency response in Fig- 
ure 17 compared with the cubic B-spline filter (1,0) in Figure 18. The 
log magnitudes of the frequency responses are plotted below; 
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Figure 17. Fiequency Response of Cubic Notch Filler 
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Figure 18. Frequency Response of Cubic B-Spline Filter 



6. Conclusions 

Designing reconstruction fihers for computer graphics applications 
requires a balanced analysis of formal quantitative properties and subjec- 
tive image quahty. There are many trade-offs, and it may be impossible 
to find a filter that yields good image quality and has good antialiasing 
properties. 



A new family of cubic filters has been analyzed, and two interesting 
filters have been found. The (1/3, 1/3) filter yields excellent image qual- 
ity, and the notch filter (3/2, -1/4) strongly suppresses postaliasing pat- 
terns. 

If derivative values can be generated by a procedural signal, an image 
with less aliasing is possible by reconstruction with Hermite interpolation 

or some other suitable filter. 

More work remains to be done. While the authors do not believe simple 
filters will be found that improve much on the cubic filters derived here, 
there are other avenues for progress. Adaptive filters might allow good 
image quality with strong antialiasing only where it is needed in problem 
areas. The effects of the reconstruction in the display and eye might be 
allowed for given models of the visual system [NET88]. 

Finally, the problem of reconstruction from nonuniform sampling is not 
entirely solved. Reasonable filters have been proposed [MIT87], but 
more analysis could be done; "Ideal" nonuniform reconstruction filters 
are known which are analogous to the sine filter used with uniform sam- 
ples. A greater challenge will be to understand the subjective issues 
mvolved in designing filters that are well suited to computer graphics. 
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Figure 12, Best-Looking Cnbic Reconstruction 



Figure 16. Using Notcli-Filter Reconstruction 



